DOCUMENT RESUME 



ED 395 957 



TM 025 093 



AUTHOR 

TITLE 



PUB DATE 
NOTE 



PUB TYPE 



Stavropoul os , Carol Susann 

Assessing Student Learning in the Arts: Building a 
Bridge between Theory and Practice. 

Apr 95 

15p.; Paper presented at the Annual Meeting of the 
American Educat i onal Research Association (San 
Francisco, CA, April 18“22, 1995). 

Reports - Evaluative/Feasibility (1A2) — 
Speeches/Conference Papers (150) 



EDRS PRICE 
DESCRIPTORS 



MFOl/PCOl Plus Postage. 



*Art Education; Cognitive Processes; Criteria; 
*Educa t i onal Assessment ; ’'Learning ; Outcomes of 
Education; Profiles; Student Evaluation; Test 
Construction; *Test Use; ’‘Theory Practice 
Relationship; ’’‘Visual Arts 



ABSTRACT 



Recent research in the visual arts has resulted in 



the deve 1 opment of a range of me thods for assessing student learning. 
Theoretically based in cognitive conceptions of learning, assessment 
systems discussed in this paper have been applied in practice. 
Possibilities for the art teacher to apply such assessment 
methodologies directly in the classroom setting are numerous. In 
addition, the assessment methodologies introduced can be employed by 
researchers seeking answers to questions regarding students' learning 
in the arts. A criterion-based rubric for art education referred to 
as the Diagnostic Profile (C . S. Stavropoul os , 1992) provides a way 
to assess written statements about art analytically. Several studies 
have shown the Diagnostic Profile to be a valid and reliable 
assessment tool. It shows the relationship between instructional 
outcomes and assessed outcomes and provides comprehensive feedback 
through multiple measures over time. (Contains 28 references.) 



(Author/SLD) 




Reproductions supplied by EDRS are the best that can be made 
from the original document. 




ERIC 



72/ ,5 ED 395 957 



1 



1 



ASSESSING STUDENT LEARNING IN THE ARTS: BUILDING A BRIDGE BETWEEN THEORY AND 



U S. DCPASTMCNT Of EDOCAT)ON 

OH>ca ol Educ«IK>n«l R««Mrcn arxl ImprovamanI 

EDUCATIONAL RESOURCES IN'^ORMATlON 
/ CENTER (ERIC) 

0^his document haa b«an raprodocfto »s 
#eceivad Jrom iha ur ofganijatior, 

o/iQinaling it 

□ Minor char^ges nave f)«en matJe to improve 
reproduction quality 



PRACTICE 
by Carol Susann Stavropoulos 
THE UNIVERSITY OF GEORGIA - ATHENS 



PERMISSION TO REPRODUCE THIS 

material has been granted by 



a Pomtsot view or opmionsstated init^iSdoCu- 
ment do not necaasanly reprasent oHiCial 
OERl position or policy 



ABSTRACT 



TO THE EDUCATIONAkL RESOURCES 
information CEN'tER tERiCr 



Recent research in visual arts has resulted in the development of a range of methods 
for assessing student learning. Theoretically based in cognitive conceptions of 
learning, assessment systems discussed in this paper have been applied in practice. 
Possibilities for the art teacher to apply such assessment methodology directly in the 
classroom setting are numerous. In addition, the assessment methodologies 
introduced can also be employed by researchers seeking answers to questions 
regarding students' learning in the arts. 



During this age of educational reform the arts continue to take a back seat to other content 
areas despite increasing theoretical and philosophical support. One reason reform leaders and 
educational traditionalists relegate the arts to the background of education is the persistent myth that 
learning in the arts cannot be assessed. The main objective of this article is to debunk that myth and 
build a bridge between theory and practice. Specifically, evidence will be provided to demonstrate that 
learning in the arts can and is being assessed, and assessment in and through the arts expands 
teaching, learning, and program options. 




PROMISING THEORETICAL FOUNDATIONS FOR ASSESSING STUDENT UNDERSTANDING IN THE 

ARTS 

Promising theoretical foundations for assessing degrees of student understanding in the arts 
are emerging. One such model focuses on the students' knowledge base and application of 
knowledge-seeking strategies. Koroseik (1993) defines the knowledge b9S6 as all the accumulated 
knowledge, skill, and experience a student currently possesses, including what the learner already 
knows about the material being studied" (p. 21). Koroseik defines kpQvyledfle-seekinq strategies as 
"the cognitive steps a student takes to construct new understandings, to seek new knowledge, and to 
apply previously acquired knowledge, skill, and experience (p. 21). While classical models of 
assessment tend to focus on the student's knowledge base, researchers (Efiand, 1990, Koroseik, 
1992-1993, 1993; Koroseik, Short, Stavropoulos, & Fortin, 1992; Parsons, 1990) contend that 
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assessment of art understandings should also focus on strategies learners use to organize knowledge. 

Transfer 

Transfer can be the result of employing knowledge-seeking strategies to make connections 
between the characteristics of the artworks and the student's accumulated knowledge or knowledge 
base (Koroscik et al., 1992, Koroscik, 1992-1993, 1993). The organization of the connections between 
the learner's knowledge base and the characteristics of the work of art can vary along a continuum 
from simple to more complex in structure (Koroscik, 1993). The connections students make when 
viewing works of art can determine whether they arrive at lower-order understandings, higher-order 
understandings, and misunderstandings. Understandings of art may, therefore, be said to vary in 
degrees of cognitive complexity, as well as accuracy (Efiand, 1990; Nickerson, 1985; Parsons, 1987, 
1990; Prawat, 1989). 

Low-road transfer and lower-order understandings. Perkins and Salomon (1987) characterize 
the most common form of transfer as low-road transfer. The process of low-road transfer involves 
lower-order search strategies such as application of familiar vocabulary, recall of facts, correct 
description of instances, and/or memorization. These knowledge-seeking strategies tend to occur as 
the automatic consequence of mental practice, and are familiar operations performed intuitively and 
automatically (Perkins & Salomon, 1987; Salomor & Perkins, 1989). 

High-road transfer and hioher-order understandings. High-road transfer as defined by Perkins 
and Salomon (1987), implies "deliberate mindful efforts to represent principles at a high level of 
generality, so that they subsume a wide range of cases" (p. 288). High-road transfer is a controlled and 
non-automated process which demands greater mental effort (Perkins & Salomon, 1987; Salomon, 
1883; Salomon and Perkins, 1989). According to Salomon and Perkins (1989), when mindful 
processes are evoked, an obvious response might be withheld in favor of a closer examination of the 
underlying meaning. Looking closer at a work of art facilitates alternative strategies, choices, and 
connections, which can lead to the construction of new structures of meaning. Constructing meaning 
through high-road transfer is a higher-order skill which promises deeper understandings and greater 
retention (Efiand, 1990; Perkins & Salomon, 1987; O'Neal, 1992; Salomon & Perkins, 1989). 
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ASSESSMENT STRATEGIES BASED ON COGNITIVE CONCEPTIONS OF LEARNING 
There has been a scarcity of assessment procedures and/or techniques that adequately 
distinguish between students' lower-order and higher-order understandings of art. In the following 
sections, assessment strategies in terms of writing about works of art will be discussed; and both 
analytic and holistic methodologies for assessing lower-order and higher-order understandings will be 



introduced. 

Assessment and Writi n g About Works of AQ 

Advocated by an increasing number of art educators and researchers, writing and talking 

about works of art encourages students to develop their verbal and perceptual skills. Further, writing 
about works of art is an effective way for students to arrive at deeper understandings of the meanings 
of works of art (Barrett, 1994; Getty Center for Education in the Arts, IGCEAl, 1985; Tollifson, 1990; 

Wilson, 1988). 

Koroscik et al. (1992) found open-ended writing tasks were useful in assessing students' 
existing knowledge and their abilities to employ appropriate knowledge-seeking strategies. Since 
transfer is visible when students apply thinking skills in writing about works of art (GCEA, 1994; 
Koroscik, et al., 1992), assessment strategies discussed in this article focus on students written 

Statements concerning works of art. 

"An integrated and active view of student learning requires the assessment of holistic and 
complex perlmmance- (Herman, Aschbacher, 8. Winters, 1992). An holistic and analytical assessment 
strategies derived from cognitive conceptions of learning have been developed for use in the field of an 
education. Piloted and field-tested in actual classrooms, these assessment strategies are now 
available for use by researchers and classroom teacher? (Stavropoulos, 1992, in press). 

Anaivtir.al nf Verbal Statements about Works gf.Arl; 

A criterion-based rubric for art education referred to as the DisqnQstiC Profile (Stavropoulos, 
1992) provides a means to analytically asses', written statements about works of art. Studies have 
shown the ni.nnn.tir-. Profile to be both a valid and reliable assessment tool, and has been well- 
received by art teachers and other professionals (GCEA, 1991; Stavropoulos, 1992, 1992-1£93a, in 
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press). 

Diagnostic profiles represent the interplay between the students' knowledge base and their 
application of knowledge-seeking strategies. Categories within the Diagnostic Profi l e account for 60 
possible learning outcomes within the formal, descriptive, interpretive, and historical dimensions of an 
understanding. Using these dimensions, art teachers can effectively discriminate between students' 
higher-ordt' and lower-order understandings, and misunderstandings. The final assessment provides 
teachers with a characterization of the students' understanding. From this profile, the teacher can 

diagnose learning constraints and learning attributes. 

Content validity studies indicate that a Diagnostic Profile assessment can be applied to a 
variety of data (Stavropoulos, 1992). These might include written statements about a single work of art 
or multiple works of art, written statements that have been verbally cued, and verbal statements that 

have been transcribed. 

Content validity studies also found assessment categories within the . PiggncstiC Prcf ilg 
applicable to statements prompted by a variety of artforms. For instance, 98% of the categories could 
be applied to verbal and written st- .ements stimulated by Western art and fine art, and that 90% could 
be used with statements stimulated by Non-Western art. In addition, the Digqnostic Praf ila was highly 
rated for its applicability to written and verbal remarks concerning popular arts, folk art, crafts, antiques 
and heirlooms, and cultural artifacts. 

Holistic Assessment of Verbal Statements abou t Wgrkg gf Art 

Holistic scoring of written statements about works of art is easier and less time intensive than 
analytic assessment. Student learning outcomes can easily be screened holistically according to 
formal, descriptive, interpretive, and historical content with the Diagnostic PrQf ilfi- Moreover, lower- 
order and higher-order understandings can be effectively distinguished through an holistic application 
of the niaonnstic Profile . For example by comparing written statements to sample lower-order and 
higher-order responses, teachers can match the overall quality of a student response to the sample it 
most clos 5ly resembles. 

Holistic application of the Diagnostic Profile also enables one or more units within a student's 
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response to be scored. Scoring units within a written response are assigned a weight of "3" (high-order 
outcome), "2" (outcome that falls somewhere in between a lower-order and higher-order outcome), "1" 
(lower-order outcome), or "0" (insufficient clarity to allow rating). With this holistic version of the 
Diagnostic Profile, teachers and researchers not only distinguish between higher-order, lower-order 
understandings - they can also tally in what dimensions these learning outcomes occur. Judgments 
are prompted with this method, and a consensus can be formed when more than one rater participates 
in the scoring session. 

Holistic assessment with the Diagnostic Profile provides a quick overview of student learning 
and achievement. Holistic assessment also provides formative feedback, which can alert the teacher 
or researcher to both the students' strengths and/or obstacles to learning. 

BRIDGING THEORY AND PRACTICE THROUGH APPLICATION STUDIES OF THE DIAGNOSTIC 
PROFILE ALTERNATIVE ASSESSMENT METHODOLOGY 

Patterns resulting from the application of holistic and/or analytic alternative assessments 
provide useful feedback about the strengths and weaknesses of individual students, classroom 
instruction, and the overall educational program. Analysis of patterns that emerge in studies can assist 
teachers, administrators, and researchers in diagnosing attributes and difficulties in learning. 

Theory and practice will be bridged through the citation of several application studies 
implementing both analytic and holistic versions of the Diagnosti c Profile alternative assessment 
methodology. The studies discussed in the following sections focus on (a) the relationship between 
instructional outcomes and learning outcomes as assessed with the Diagnostic Profile , and (b) the use 
of multiple Diagnostic Profile assessments in distinguishing and characterizing the effects of 
instruction over periods of time. 

The Diagnostic Profile and the Relationship Between Instruction and Learning Outcomes 

According to Herman, Aschbacher, and Winters (1992) the cognitive learning theorists 
approach to knowledge acquisition supports the need to integrate assessment methodologies with 
instructional outcomes (p. vi). Therefore, it is crucial that alternative assessments (a) reflect the 
outcomes that occur through classroom instruction, and (b) distinguish more cognitively complex 
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learning outcomes from less sophisticated learning outcomes. The relationship between instruction 
and learning outcomes are investigated in two Diagnostic Profile application studies that follow. The 
first study focuses on the relationship between observed discipline-based art education (DBAE) 
instructional outcomes and learning outcomes assessed with the Diagnostic Profile . The second study 
investigates the impact of a depth of understanding instructional approach on students understandings 
of artworks. 

Relationship between DBAE instructional outcomes and learning outcome. The principal 
investigator observed a class of 3rd/4th-grade students and a class of 8th ;jrade students and the 
instruction they received over a period of nine weeks. Both the 3rd/4th-grade art teacher and the 8th- 
grade art teacher taught concepts of DBAE, however, a sharp contrast in how they operationalized 
DBAE was apparent. 

Observations of the 3rd/4th-grade class showed that students received instruction that 
prompted lower-order thinking skills. Students writing samples were expected to be heavily influenced 
by the art teacher's instruction which included (a) a lecture format where students viewed slides in a 
dark room as the teacher read a script, and (b) questioning and recitati' of historical facts about 
artists and works of art and vocabulary terms. 

Observations of the 8th-grade class revealed that students received instruction that 
encouraged higher-order thinking skills. For instance, these 8th-grjde students (a) conducted their 
own research, (b) expressed their ideas through a range of writing activities, (c) constructed 
arguments and provided support of their stance with reasons, and (d) participated in art criticism 
activities and aesthetic inquiry. The teacher's implementation of these instructional strategies 
encouraged students to become active participants in their own learning. 

Qualitative data in the form of written statements was then gathered from both the 3rd-grade 
and Sth-grade students at the end of the observational periods. These written statements were 
analytically scored with the Diagnostic Profile , and compared to the DBAE instructional methods 
implemented by the respective art teacher. 

Results of the study demonstrate an extremely strong relationship between instruction received 
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by students and the assessment of student understanding with the Diagnostic Profile- As predicted, 
the Diagnostic Profile analysis showed (a) a preponderance of low'er-order understandings contained 
in the written statements of the 3rd/4th-grade students, and (b) the majority of the 8th-grade students 
written statements exhibited outcomes reflective of higher-order understandings. Furthermore, this 
study provides convincing evidence of the Diagnostic Profile's effectiveness in discriminating lower- 
order understandings from hiaher-order understandings (Stavropoulos, 1992, 1992-1 993b). 

Effects of a Depth of Understanding Model on Student Understanding 

"Any study investigating factors which may influence the understanding of visual art 
information must initially be concerned with the way in which visual (pictorial) information is cognitively 
processed" (Short, 1993, p. 1). Koroscik (1982) developed a model which suggests that art 
understandings can be enhanced by focusing student discussions on formal qualities first, followed by 
descriptive content and interpretation. 

Short (1993) incorporated Koroscik's "depth of understanding" model in a high school studio 
curriculum to provide students a foundation for discussion and writing activities concerning works of 
art. The purpose of this study was to determine whether studio curriculum based upon Koroscik’s 
depth of understanding model would; 

1 . improve students ability to write about works of art; 

2. improve students understanding of works of art: 

3. improve students ability to interpret works of art; 

4. challenge students' misconceptions that artworks are based on technique alone. (Short, 1 993) 

To probe the effectiveness of a depth of understanding approach on student understanding, an 

experimental study design incorporating a pre-test and post-test measure. Four experimental groups 
were randomly selected. Three intact beginning level high school drawing classes participated in the 
study (n = 54). in addition, an intact class of advanced placement English majors with no previous art 

experience served as the control group (n = 28). 

Two treatment group received the normal series of studio lessons in drawing. The same 
drawing lessons were provided to the third treatment group, but were accompanied with verbal 
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interaction and writing activities about artists and works of art; and organized according to a depth of 
understanding model. The depth of understanding model provided curricular emphasis on dimensions 
of art understanding in the following sequence; formal qualities, description, and interpretation. 

To demonstrate the effects of the semesters instruction, pre-test and post-tests directed 
students from each experimental group to analyze a work of art, and offer an interpretation. These 
written responses constituted qualitative data which required analysis. To characterize the differences 
in formal, descriptive, and interpretive historical understanding of the art stimuli, students pre and 
post-test writing samples were analyzed with the Diagnostic ProjiLq. 

As a result of the treatment, students' overall understanding of artworks improved dramatically. 
The Diagnostic Profile assessment showed student gains in the amount of information they wrote, while 
demonstrating substantial growth in formal and descriptive understanding of the work of art. 
Additionally, a significant increase was noted in students' understanding of interpretive qualities in the 
experimental group that received instruction according to the depth of understanding model. 

The Diagnostic Profile and Multiple Measures Taken Over a Peri- jd pf Time 

Herman, Aschbacher, and Winters (1992) contend "assessment systems that provide the most 
comprehensive feedback on student growth include multiple measures taken over time" (p. vi). In the 
three application studies that follow, multiple measures of Diagnostic Profile were employed in (a) 
distinguishing the effects of a year long writing intensive DBAE curriculum; (b) a three year longitudinal 
study of honors students' reporting of artistic events; and (c) characterizing the effects of interactive 
media on the art criticism abilities of students enrolled in art survey courses. 

Assessing writing intensive DBAE curriculum. In distinguishing the effectiveness of a writing 
intensive DBAE curriculum, a quasi-experimental study compared students' written responses to a 
work of art. Three intact 4th-grade classrooms received either a writing-intensive approach to DBAE, a 
non-writing approach to DBAE, or a traditional studio-based approach to art education throughout the 
school year. Pre-test and post-test qualitative data in the form of written statements about an artwork 
were collected from each group at the beginning and end of the academic year. Pre-test and post-test 
data were scored holistically with the Diagnostic Profile . 
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The Diagnostic Profile analysis enabled comparisons of pre-test and post-test data within each 
experimental group. Both the non-writing DBAE approach and the DBAE writing intensive approach 
have impacted student ability to write about the art stimuli. However, the writing intensive approach to 
DBAE influenced students' ability to effectively communicate understanding a work of art most 
significantly. While studio-based art instruction provided fundamental experiences with art media and 
technique, students in the control group did not show any marked growth in understanding the art 
stimuli (French, 1992; Stavropoulos, 1994). 

Longitudinal study of honors students' reporting of artistic events . This study examines the 
effects of art exposure in combination with required critical written responses on honor students 
understandings of the arts. This three year long longitudinal study serves as a program evaluation of 
an honor's program in art at a small Southern college. The experimental component of the study 
explores the contribution of the honors' program to the aesthetic orientation of college students 
enrolled in academic honor's courses, and consists of three treatment groups. The researcher 
projected that involvement in an honors program specially designed to address professional 
engagement in various discipline areas (arts and sciences, business, education, and religion) would 
influence students' critical written responses about arts events. The study has four specific evaluation 
components: 

1 . Are students discerning? Are they able to select relevant information about which to write, and 

are they able to appropriately categorize this information? 

2. Are students appreciative of the art events they report? 

3. Are students well informed about the events they attend and discuss? 

4. Do students perceive themselves as "critics"? 

In the first year of the study, intervention included student feedback regarding achievement 
levels they attained on written reports across a range of art events. This feedback centered around 
evaluation components 1, 2, and 3. In the second and third years of the study, intervention consisted 
of stressing students' perception of themselves as "critics" in writing reports about art events. A 
random sample of written reports (n = 62) were selected from the stratified data pool collected over this 
three year period (N-^300). The Diagnostic Profile assessment of written reports over this three year 
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period provides a descriptive and diagnostic analysis of honors students' advancement in aesthetic 
orientation (Diket & Stavropoulos, in press). Regression analysis was also performed on variables 
such as length of time students were enrolled in honors program, departmental associations, 
traditional vs non-traditional student types, age, ethnicity, and grade point average. The hypothesis 
was confirmed - the honors program and written responses exhibit a strong positive relationship. 

Multiple time series analysis of effects of interactive media. The effects of interactive 
multimedia on the art critical abilities of art students is the focus of a study currently underway at the 
University of North Texas. Learning through interactive multimedia provides for a level of visualization 
not possible with traditional slide study. A multitude of artworks can be studied in rich interdisciplinary 
contexts enhanced by audio, text, animation, full-motion video, and graphics. 

The study will compare the effects of two levels of an interactive multimedia program, versus 
traditional slide study on the writings of students enrolled in art history survey courses. The following 
research questions will be addressed through Diagnostic Profile analysis of students' written 
statements about a key artwork: 

1 . Will interactive multimedia prove to be a more effective instructional aid than slide study m 
understanding a work of art? 

2. What kind of interactive multimedia system is most effective in promoting critical analysis? 

3. To what levels of understanding (higher-order, lower-order, understanding or 
misunderstanding) will use of interactive multimedia contribute? 

4. To which dimensions of art understanding (formal, descriptive, interpretive, historical) will use 
of interactive multimedia contribute? (Cason, 1995) 

To better understand the educational impact of interactive multimedia on art learning, the study 
will incorporate a counter-balance design. A total of 70 undergraduate art students will be randomly 
bv. ' ’.ed to participate in the study. Of these undergraduate students, 35 will be assigned to an 
experimental group, and 35 will be assigned to a Control group. An interactive multimedia program, 
authored by Audio-Visual Connection (AVC) will be utilized by the experimental group. Students in the 
experimental group will access the AVC interactive multimedia program in an Education Computer Lab 
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as an additional study resource to the survey course. The supplemental study resource for the control 
group will consist of slides of art works shown in the lecture and the text. After the first measure, the 
groups will share treatments to control for the confounding variable computer familiarity. On the 
second measure, the experimental group will use the more interactive multimedia program authored by 

AVC. 

At intervals following specific units of instruction, writing samples stimulated by the key artwork 
will be collected from both the experimental and control groups. These writing samples were analyzed 
with the niannnsTic Profile in order to characterize incremental effects of interactive media on student 
understanding, as well as cumulative gains in student understanding over the course of instruction 

(Cason, 1995). 

CONCLUDING REMARKS 

In this article, promising theoretical foundations for assessing student understanding in the arts 
were discussed. Both holistic and analytic assessment strategies, based in cognitive conceptions of 
learning, were introduced. Finally, theory and practice was bridged through the citation of research 
studies where alternative assessment methodology was applied. These various studies demonstrate 
the effectiveness of the Diagnostic Profile in (a) showing the relationship between instructional 
outcomes and assessed outcomes, and (b) providing comprehensive feedback through multiple 
measures over time. 

The importance of the development of alternative assessment strategies lie in future 
applications. "Quality assessment provides substantial data for making informed decisions about 
student learning" (Herman, Aschbacher, & Winters, 1992, p. vi). The Diagnostic Pr gfilfi can provide an 
assessment of the cognitive learning outcomes displayed by primary, elementary, and secondary 
school-aged students, and undergraduate and graduate students. 

Findings and data generated through application of the Diggnostic Prpfil .£ can inform teachers, 
administrators, and art education researchers about emerging programs in art education. Since the 
nip.nnni.tic Profile enables random sampling from populations, results provided by large-scale studies 




can (a) assist in designing more effective curricula, (b) demonstrate to parents that art programs make 
a difference, and (c) assist in advocacy efforts by providing comparative data. Such efforts might help 
secure funding for future research in art education and assist policy-makers in determining more 
epuitaPle allocations of educational resources. 
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